An Empirical Study of Test Generalization in NUnit Framework

Owing to the significance of unit-testing in software development life-cycle, there are several existing approaches on automating the generation of conventional unit tests. However, these approaches often fail to generate a set of test cases that can ensure a high code coverage and manually writing such test cases is a labor-intensive task. To address these preceding issues with conventional unit tests, Parameterized Unit Tests (PUT) are introduced where programmers can describe the expected behavior or specifications. Pex, an automatic test generation tool, accepts these PUTs and generates a set of conventional unit tests that tries to ensure a high code coverage. In this paper, we present an empirical study to analyze the benefits of PUTs. We use two phases in the process of converting conventional unit tests into PUTs: generalization and writing new PUTs. The generalization phase refers to the transformation of existing conventional unit tests into PUTs, whereas writing new PUTs refers to the addition of new PUTs to cover the code under test that is not covered by the generalization phase. In our empirical study, we used an open source C# project called NUnit to study the benefits of test generalization. In our study, we found that generalization has increased the code coverage by 9.68% (on average) and also detected 7 new defects. Furthermore, writing new PUTs resulted in a total increase in the code coverage achieved with conventional unit tests